Video compositing systems for providing interactive entertainment

ABSTRACT

An interactive video compositing device includes a chroma-key mixer, video switcher and control circuitry. The chroma-key mixer generates a composite image by combining a real-time image, such as one captured by a video recorder, with a prerecorded video image, such as a movie. The composite image includes the modified real-time image superimposed, or overlaid, onto the prerecorded image. The video switcher automatically selects either the composite image or the prerecorded image to be output to a display. The control circuitry controls the video switcher and other outputted signals based on data file information that corresponds to content of the prerecorded image or media. For example, the data files may contain information relating to the presence (or absence) of a particular character in a movie scene, thus allowing for the output and display, at appropriate times, of the real-time composite image instead of the prerecorded image.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/256,374, filed Oct. 22, 2008, which is a continuation of U.S. patentapplication Ser. No. 10/836,729, filed Apr. 30, 2004, now U.S. Pat. No.7,528,890, issued May 5, 2009, which claims the benefit of priorityunder 35 U.S.C. § 119(e) of U.S. Provisional Application No. 60/467,480,filed May 2, 2003, and entitled “INTERACTIVE SYSTEM FOR REAL-TIME CHROMAKEY COMPOSITING,” each of which is hereby incorporated herein byreference in its entirety to be considered part of this specification.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Preferred embodiments of the invention relate to a system and method forsignal compositing, and more particularly, to a system and method forinteractive video compositing.

2. Description of the Related Art

Interactive entertainment is a popular leisure activity for peopleacross the globe. One favorite activity for many is karaoke, whichtemporarily turns lay persons into “stars” as they sing the lyrics to afavorite song. Karaoke machines play the music of a selected song whilesimultaneously displaying the song lyrics to a user, thus allowing theuser to sing along with the background music.

Another favorite leisure activity for millions is watching movies.Billions of dollars are spent each year on ticket sales for new boxoffice hits. In addition, an increasing number of people are beginningto view movies at home. One popular technology that has been recentlyadopted is the digital video disk (DVD) technology. Movies are recordedin digital format onto a disk that allows for repeated viewings at theconvenience of the user. The sound and picture quality of DVDs, alongwith the convenient features that accompany digital media, have madeDVDs a popular replacement to standard video cassette technology.

However, movie watching has predominantly been a passive activity. Thatis, one either travels to a theater or inserts a disk into a DVD playerand sits back to watch the movie. Though one may watch the same movierepeatedly, each time the same characters appear and recite the samelines and perform the same actions. As of yet, one has not been able toact out his or her favorite scene while appearing to be in the movieitself in a real-time manner.

SUMMARY OF THE INVENTION

Accordingly, there is a need for an interactive system that allows forvideo images to be inserted into a prerecorded video, such as a movie.In one embodiment of the present invention, chroma-key technology isused to produce a composite video image, wherein it appears that atarget image, such as one captured by a video camera, is superimposed inreal time onto a prerecorded image. The compositing process is managedby internal circuitry and data files that direct when and how thereal-time image is inserted into the prerecorded image.

In one embodiment, an interactive video compositing device includes amixer, video switcher and control circuitry. The mixer utilizeschroma-key technology to remove a chroma element from a real-time videoimage. The mixer combines the modified image with a prerecorded videoimage to form a composite image, which includes the modified real-timeimage overlaid on the prerecorded image. The video switcherautomatically selects either the composite image or the prerecordedimage to be output to a display. This selection is managed by thecontrol circuitry and is based on data file information that correspondsto the content of scenes of the prerecorded image. For example, the datafiles may contain information relating to the presence (or absence) of aparticular character in the movie scene, thus allowing for the output ofthe real-time composite image instead of the prerecorded image atappropriate times.

In another embodiment of the invention, a system for real-time,chroma-key compositing comprises a first input structured to receive afirst video signal; a second input structured to receive a second videosignal; a chroma-key mixer coupled to the first input and to the secondinput, wherein the chroma-key mixer is structured to output a compositesignal comprising portions of the first video signal and portions of thesecond video signal; a data file associated with the first video signal,wherein the data file has at least first reference data and secondreference data; and a switcher that selects between outputting thecomposite signal and outputting the first video signal, wherein theswitcher outputs the composite signal at a time identified by the firstreference data, and wherein the switcher outputs the first video signalat a time identified by the second reference data.

In another embodiment, a reference data system used for videocompositing comprises first reference information associated with afirst video signal; second reference information associated with thefirst video signal; and a video switcher configured to switch betweenoutputting either the first video signal or a composite signal thatincludes portions of the first video signal and portions of a secondvideo signal, wherein the video switcher outputs the composite signal ata time identified by the first reference information, and wherein thevideo switcher outputs the first video signal at a time identified bythe second reference information.

In another embodiment of the invention, a software program comprisescode for controlling the superimposing of a real-time image onto aprerecorded image, wherein said code identifies portions of theprerecorded image during which the real-time image is to be superimposedonto the prerecorded image.

In another embodiment, a system for real-time video compositingcomprises a means for generating a composite signal by chroma-key mixinga first video signal and a real-time second video signal; and a meansfor switching between the selection of the first video signal and theselection of the composite signal, wherein the means for switching iscontrolled according to reference information stored in at least onedata file that relates to the first video signal.

In another embodiment, a method for video compositing comprisesreceiving a first video signal having multiple frames; receiving asecond video signal; creating a composite signal by chroma-key mixingthe first video signal and the second video signal; and selectivelyswitching, with a video switcher, between outputting the first videosignal and outputting the composite signal, wherein said selectiveswitching is performed automatically and is based on a control signalreceived by the video switcher, wherein the control signal is derived atleast in part from prerecorded data that corresponds to frames of thefirst video signal.

In another embodiment, a method for coordinating the display of multiplesignals comprises receiving a first signal; receiving a second signal;forming a composite signal comprising portions of the first signal andportions of the second signal; receiving with a switcher the firstsignal and the composite signal; receiving reference data thatcorresponds to portions of the first signal; and switching betweendisplaying the first signal and displaying the composite signal, whereinthe timing of said switching is based on information contained in thereference data.

In another embodiment, a method for creating a data file used forcontrolling the superimposing of video images comprises identifying aportion of a first video image during which a second video image is tobe superimposed onto the first video image, said first video imageportion having a beginning frame and an ending frame; recording in adata file first information that identifies said beginning frame; andrecording in the data file second information that identifies saidending frame.

For purposes of summarizing the invention, certain aspects, advantagesand novel features of the invention have been described herein. It is tobe understood that not necessarily all such advantages may be achievedin accordance with any particular embodiment of the invention. Thus, theinvention may be embodied or carried out in a manner that achieves oroptimizes one advantage or group of advantages as taught herein withoutnecessarily achieving other advantages as may be taught or suggestedherein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a real-time video compositing system according to oneembodiment of the invention.

FIG. 2 is a block diagram of one embodiment of a compositor device.

FIG. 3 illustrates a user interface of one embodiment of a compositordevice.

FIG. 4 is a block diagram of an embodiment of a compositor device thatis structured to receive input from two video recorders.

FIG. 5 is a flow chart depicting one embodiment of an interactive videocompositing process.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The features of the system and method will now be described withreference to the drawings summarized above. Throughout the drawings,reference numbers are re-used to indicate correspondence betweenreferenced elements. The drawings, associated descriptions, and specificimplementation are provided to illustrate embodiments of the inventionand not to limit the scope of the invention.

In addition, methods and functions described herein are not limited toany particular sequence, and the steps or states relating thereto can beperformed in other sequences that are appropriate. For example,described steps or states may be performed in an order other than thatspecifically disclosed, or multiple steps or states may be combined in asingle step or state.

FIG. 1 illustrates one embodiment of a real-time video compositingsystem 10. The compositing system 10 is used to selectively superimposeimages. These images may comprise images that can be viewed inreal-time, prerecorded images, or a combination thereof. For example,the compositing system 10 may superimpose a real-time image onto animage that has been prerecorded, such as a movie. The compositing system10 generally comprises a compositor device 12, a video source 14, avideo recorder 16, and a display 18. In the depicted embodiment, thecompositor device 12 receives inputs signals from the video source 14and the video recorder 16 and outputs a signal to the display 18.

The term “compositing” as used herein is a broad term and is used in itsordinary sense and includes without limitation the superimposing orcombining of multiple signals, such as, for example, video and/or audiosignals, to form a composite signal. The term “compositor” refers to anydevice or system, implemented in hardware, software, or firmware, or anycombination thereof, that performs in whole or in part a compositingfunction.

The term “real time” as used herein is a broad term and is used in itsordinary sense and includes without limitation a state or period of timeduring which some event or response takes place. A real-time system orapplication produces a response to a particular stimulus within acertain response time. For example, a device processing data in realtime may process the data as it is received by the device. A real-timesignal is one that is capable of being displayed, played back, orprocessed within a particular time after being received or captured by aparticular device or system. In one embodiment, this particular time ison the order of one millisecond. In other embodiments, the particulartime may be longer than one millisecond. For example, the particulartime may be on the order of hundreds of milliseconds. In otherembodiments of the invention, the particular time may be less than onemillisecond. For example, the particular time may be on the order ofmicroseconds. In yet other embodiments of the invention, “real time”refers to events simulated at a speed similar to the speed at which theevents would occur in real life.

The video source 14 includes any device, system or technology used togenerate, receive, capture, read, supply or store video data. In anotherembodiment, the video source 14 may generate an audiovisual signal thatincludes a video portion that can be processed to produce a video signal(e.g., to produce a visual image) and an audio portion that can beprocessed to produce an audio signal (e.g., sound at a level high enoughto be heard by a human ear). For example, in one embodiment of theinvention, the video source 14 comprises a digital video disk (DVD)player. In another embodiment, the video source 14 comprises a memorythat stores data representing video content. In yet other embodiments ofthe invention, the video source 14 may comprise a device that receives avideo transmission, such as through a cable network, a satellite dish,an antenna, or a network. For example, the video source 14 may comprisea television, a video cassette recorder (VCR), a CD+G player or adigital video recorder.

In other embodiments of the invention, the compositing system 10 mayinclude multiple video sources 14, each being coupled to the compositordevice 12. In such an embodiment, the compositing system 10 may comprisea multiplexer or a switch that selects a signal from one of multiplevideo sources 14. For example, in one embodiment, the compositing system10 comprises a DVD player that reads data from a DVD and a cable boxthat receives a video transmission over a coaxial cable line. Atwo-input multiplexer may then be used to select between a signal fromthe DVD player and a signal from the cable box.

The video source 14 may be coupled to the compositor device 12 by anymedium that provides for video signal transmission. For example, thevideo source 12 may be coupled to the compositor device 12 through anRCA cable, an S-cable, a coaxial cable, Ethernet, wireless technologiesand the like. One with skill in the art will also recognize that thevideo source 14 may supply audio content along with video content. Thisaudio content may be delivered on the same or different mediums as thevideo content.

FIG. 1 depicts an embodiment of the invention wherein the video source14 is external to the compositor device 12. In other embodiments of theinvention, the video source 14 may be internal to the compositor device12. For example, the compositor device 12 may comprise a DVD player ormay comprise a memory having stored video data. In yet otherembodiments, the compositing system 10 may comprise at least one videosource 14 that is internal and at least one video source 12 that isexternal to the compositor device 12.

With continued reference to FIG. 1, the compositing system 10 includesthe video recorder 16. The video recorder 16 comprises any device,system or technology that is capable of converting real-time videoimages into an electronic signal, such as a digital or an analog signal.In one embodiment, the video recorder 16 is capable of the real-timeconversion of and transmission of video images. In one embodiment, thevideo recorder 16 is a video camera. For example, the video recorder 16may comprise a camcorder, such as an analog camcorder or a digitalcamcorder. The video recorder 16 may be coupled to the compositor device12 through an RCA cable, an S-cable, a coaxial cable, Ethernet, wirelesstechnologies and the like.

In other embodiments of the invention, the compositing system 10 mayinclude multiple video recorders 16, each being coupled to thecompositor device 12. For example, multiple video cameras may be coupledto the compositor device 12. In addition, the video recorder 12 may beinternal or external to the compositor device 12.

The compositing system 10 also comprises the display 18. The display 18receives an output signal from the compositor device 12 and converts theoutput signal to at least a video image. For example, in one embodimentof the invention, the display 18 comprises a television that is coupledto the compositor device 12 through RCA cables. In other embodiments,the display 18 may include a video projector, a monitor or the like andmay be coupled to the compositor device 12 through any medium thatprovides for video signal transmission, as has been previouslydescribed.

In other embodiments of the invention, the display 18 may also be usedto provide instructions or data to the user or users of the compositingsystem 10. For example, menu selections or command prompts may bedisplayed to the user through the display 18. In addition, dialogueprompts, such as are used in general karaoke machines may be portrayedon the display 18 so as to assist a user in reciting the appropriatelines.

In other embodiments of the invention, the compositing system 10 maycomprise multiple displays 18. The display 18 may also be internal orexternal to the compositor device 12. For example, the compositor device12 may include a screen that portrays a video image to the user. Such ascreen would allow a user to have visual feedback as to the final outputof the compositing system 10 without having to look at an externaldisplay.

In one embodiment, the compositing system 10 may also comprise a mediastorage device (not shown) that stores the signal output by thecompositor device 12. For example, the compositor system 10 may comprisea memory configured to store in digital form a copy of the output signalthat is sent to the display 18. In another embodiment, the compositingsystem 10 may output a signal only to the media storage device insteadof the display 18. In such an embodiment, the output video and audiocontent could be stored for later playback on another device. In yetother embodiments, the media storage device may be included with thedisplay 18.

FIG. 2 illustrates a block diagram of one embodiment of the compositordevice 12. The compositor device 12 allows a user to selectively overlayimages in real time onto a second video image, such as prerecorded videocontent. In one embodiment, the compositor device 12 comprises controlcircuitry 20, a memory 22, a DVD player 24, a multiplexer 26, achroma-key mixer 28, a switcher 30 and a user interface 32.

In one embodiment, the components of the compositor device 12 aremodules that comprise logic embodied in hardware or firmware, or thatcomprise collection of software instructions written in a programminglanguage, such as, for example C++. A software module may be compiledand linked into an executable program, installed in a dynamic linklibrary, or may be written in an interpretive language such as BASIC. Itwill be appreciated that software modules may be callable from othermodules or from themselves, and/or may be invoked in response todetected events or interrupts. Software instructions may be embedded infirmware, such as an EPROM or EEPROM. It will be further appreciatedthat hardware modules may be comprised of connected logic units, such asgates and flip-flops, and/or may be comprised of programmable units,such as programmable gate arrays or processors. For example, in oneembodiment, the functions of the compositor device 12 may be implementedin whole or in part by a personal computer or other like device.

It is also contemplated that the components of the compositor device 12need not be integrated into a single box. The components can beseparated into several subcomponents or can be separated into differentdevices that reside at different locations and that communicate witheach other, such as through a wired or wireless network, or theInternet. Multiple components may be combined into a single component.It is also contemplated that the components described herein may beintegrated into a fewer number of modules. One module may also beseparated into multiple modules.

The control circuitry 20 manages the operation of components of thecompositor device 12. In one embodiment, the control circuitry 20 is aspecial purpose microprocessor. In other embodiments, the controlcircuitry 20 may be implemented as an application-specific integratedcircuit (ASIC). In yet other embodiments, the control circuitry 20 maybe implemented as one or more modules, which modules may be configuredto execute on one or more processors. The modules may comprise, but arenot limited to, any of the following: hardware or software componentssuch as software object-oriented software components, class componentsand task components, processes, methods, functions, attributes,procedures, subroutines, segments of program code, drivers, firmware,microcode, applications, algorithms, techniques, programs, circuitry,data, databases, data structures, tables, arrays, variables, or thelike.

The control circuitry 20 communicates with the memory 22. The memory 22may comprise any buffer, computing device, or system capable of storingcomputer instructions and data for access by another computing device ora computer processor, such as, for example, the control circuitry 20. Inone embodiment, the memory 22 comprises random access memory (RAM). Inother embodiments, the memory 22 may comprise other integrated andaccessible memory devices, such as, for example, read-only memory (ROM),programmable ROM (PROM), and electrically erasable programmable ROM(EEPROM). In another embodiment, the memory 22 comprises a removablememory, such as a floppy disk, a compact disk (CD), a ZIP® disk, a DVD,a removable drive or the like.

In one embodiment, the memory 22 stores data files that include data orinformation regarding the content of particular media. In oneembodiment, a data file or a group of related data files containsinformation specific to a particular movie or prerecorded video footage.For example, data files for a particular movie may be referenced byusing the movie's unique serial number. The information contained in thedata files may identify scenes or segments of the particular movie orvideo that have been catalogued as being suitable for video compositing.The data files may also contain content relating to dialogue prompts forparticular characters, menu options, and other data relating to scenesavailable for video compositing.

In one embodiment, the data files contain reference information thatidentifies particular frames or points in a video source that may belater used in the compositing process. For example, in one embodimentthe reference information is used by the composting device 12 toidentify the frames of a movie scene in which a particular character ispresent. In one embodiment, the reference information may be used totrigger multiple signals for use in the compositing process. Forexample, the reference information may contain both beginning and endingreference points, wherein the beginning reference point indicates thecommencement of a particular feature in the video source, such as theentrance of a character into a scene, and wherein the ending referencepoint identifies the ending of a particular feature, such as when thecharacter leaves the scene.

The term “reference information” as used herein is a broad term and isused in its ordinary sense and includes without limitation any type, orcombination of types, of data that stores or contains informationregarding particular media. For example, as described above, referenceinformation may comprise reference points that identify scenescontaining particular characters. However, reference information is notlimited to such reference points. In other embodiments of the invention,reference information may comprise code, symbols, alphanumericinformation, or the like that represent a song, a particular event, or aparticular image that is contained or represented in particular media,such as an audiovisual signal.

In one embodiment, data files may be stored in the memory 22. Forexample, data files stored in the memory 22 may contain informationrelating to the individual frames of a particular movie, such as StarWars. In one embodiment, the preprogrammed data files associated withStar Wars would identify scenes in the movie that have been selected asbeing suitable as a background scene for video compositing. The datafiles may identify when a particular character, such as Darth Vader®, ispresent in a specific scene. This data file information allows for theuser to chose between pre-selected scenes and facilitates the generationof a signal by the control circuitry 20 to be used in coordinating andcreating a composite video signal. The data files may be preprogrammedin the memory 22. In other embodiments, the data files may be latersaved in the memory 22 by the user.

In other embodiments, the data files need not be stored in the memory22. The data files may be generated in real time, generated on the fly,derived or received from an external source or device, or generated bythe compositor device 12 itself. For example, data files for particularmedia could be downloaded from the Internet, transferred from aremovable storage medium, or even programmed by the user. The data filesmay also be embedded in a closed-caption signal or be saved on a DVD,such as in a bonus material section, that contains the correspondingmovie.

Although the memory 22 is depicted as being external to the controlcircuitry 20, in other embodiments of the invention, the memory 22 maybe internal to the control circuitry 20. For example, the memory 22 mayexist as a cache in the control circuitry 20. The memory 22 may also beexternal to the compositor device 12. For example, in one embodiment,the memory 22 comprises an external hard drive. The memory 22 may alsocomprise multiple memory devices for storing data or information.

The DVD player 24 is one embodiment of a video source for the compositordevice 12. The DVD player 24 functions as a general purpose DVD playerand outputs video and audio content stored on a DVD to the multiplexer26. The DVD player 24 also includes a counter that adjusts itself basedon which frame of the DVD is being read. For example, the DVD countermay correlate each frame of the DVD with a specific time code relatingto the media stored on the DVD. This counter enables the DVD player 24to jump to, identify or read specific frames of video content stored onDVDs. The DVD player 24 also reads DVD serial numbers so as to identifythe media content contained by the particular DVD and may communicatethe serial number to the control circuitry 20.

The functioning of the DVD player 24 may be controlled by the controlcircuitry 20, which loads appropriate data files from the memory 22. Forexample, when a Star Wars DVD is placed in the DVD player 24, the DVDplayer reads the serial number of the DVD and communicates the number tothe control circuitry 20. The control circuitry 20 uses the serialnumber to find the appropriate data files stored in the memory 22. Thedata files identify the media content of the DVD as being the Star Warsmovie and also identify which scenes, or frames, are to be played by theDVD player 24. In this way, the control circuitry 20 is able to managethe functioning of the DVD player 24.

In one embodiment of the invention, the DVD player 24 also reads DVDsthat contain data as opposed to a movie or a video. For example, in oneembodiment, the DVD player 24 is used to read DVDs that contain datafiles that are associated with several movies or videos, which datafiles may be copied to the memory 22.

With continued reference to FIG. 2, other video sources, such asexternal video sources, may communicate with the compositor device 12.In one embodiment, the multiplexer 26 is configured to accept signalsfrom multiple external sources as well as from the DVD player 24. Forexample, the multiplexer 26 is shown as being structured to receivesignals from the DVD player 24, a cable network, an antenna and asatellite. In other embodiments, the multiplexer 26 may be configured toreceive fewer or more signals. For example, multiplexer 26 may beconfigured to receive a streaming video over the Internet or data from acable box. The multiplexer 26 may also be configured to receiveauxiliary signals from an external DVD player or a VCR.

The multiplexer 26 is structured to select one of multiple inputsignals, the selection being based on a control signal. In oneembodiment, the control circuitry 20 supplies the control signal to themultiplexer 26. In another embodiment, the multiplexer 26 automaticallyselects the signal from the DVD player 24 when a DVD is inserted thereinand selects the signal from the other available signals when no DVD ispresent in the DVD player 24. For example, in one embodiment, the usermay input a selection through the user interface 32.

As shown in FIG. 2, one embodiment of the multiplexer 26 outputs theselected signal to the chroma-key mixer 28 and the switcher 30. In otherembodiments of the invention, other switching devices or routers may beused in place of the multiplexer 26 to select between multiple inputsignals and to communicate the selected signal to other components. Inother embodiments of the invention, the multiplexer 26 may receivesignals from other video sources. The multiplexer 26 may also be coupledto more or fewer video sources than are depicted in FIG. 2.

The term “chroma-key” as used herein is a broad term and is used in itsordinary sense and refers to without limitation a system, device, orprocess that is used to create an effect wherein at least one color orhue in a video image is eliminated or substituted for with a differentimage. For example, a chroma-key technique, also referred to as colorseparation overlay, may utilize a mixer or like device to substitute acolor, such as blue or green, in one video image for portions anothervideo image.

The chroma-key mixer 28 receives, as inputs, signals from themultiplexer 26 and from the video recorder 16. The chroma-key mixer 28processes these two input signals to form an output composite signal,which is communicated to the switcher 30. The chroma-key mixer 28 mayalso receive control signals from the control circuitry 20 or from theuser interface 32.

In one embodiment of the invention, the chroma-key mixer 28 is used tocreate special visual effects that utilize the combination of two videosignals to produce one composite image. In particular, the chroma-keymixer 28 is used to produce a composite image wherein it appears thesubject from one video source, such as footage being captured by a videocamera, is inserted into the footage from another video source, such asa movie on a DVD. This mixing by the chroma-key mixer 28 may beaccomplished in real time.

In one embodiment, the chroma-key mixer 28 produces a composite image bysubtracting a chroma element or elements from the real-time image, suchas an image being captured by the video recorder 16. The chroma elementcomprises at least one color that has been pre-selected or that isselected by the user, which color is used in the background for thevideo-recorded image. Upon receiving the real-time signal, thechroma-key mixer 28 removes the chroma element (the background) from thevideo recorded image, leaving only the image of the target subject.

For example, in so-called “green screening,” a target subject ispositioned in front of a solid green screen. The image of the targetsubject is then captured by a video recorder and transmitted as a signalto the chroma-key mixer 28. The chroma-key mixer 28 subtracts the chromaelement (green) from the video recorder signal. This leaves only theimage of the target subject along with “blank” portions where thereal-time image had contained the chroma element. The chroma-key mixer28 then replaces the subtracted, or blank, portions of the real-timeimage with portions of the image contained by the signal from themultiplexer 26, which signal contains scenes from a movie on a DVD. As aresult of the signal processing, it appears that the target subjectimage, which is a real-time image, is present in a movie or otherprerecorded video, thus forming a composite image. In other words, thecomposite image is made up of at least two video components: aforeground image, which consists of the non-chroma element portions ofthe video recorder signal, and a background image, which consists of thesignal received from the multiplexer 26.

Though this example describes the functioning of one embodiment of thechroma-key mixer 28, one with skill in the art will recognize that thereexist other processes that may be used to produce a composite signal.For example, the chroma-key mixer 28 may directly substitute portions ofthe video source signal for the chroma-element portions of the real-timevideo signal. In another embodiment of the invention, the chroma-elementportions of the real-time video signal are made transparent by thechroma-key mixer 28. This allows the non-chroma element portions of thereal-time video signal to be layered on top of the video source signalto create the composite image.

Though the example given above has been in reference to an embodiment ofthe invention utilizing green as the chroma element, other colors may beused. For example, blue or red may be designated as the chroma element.In addition, multiple shades of the same color may be identified aschroma elements, allowing for a finer tuning of the composite image bythe chroma-key mixer 28. For example, in one embodiment of theinvention, the user is able to select portions (or colors) of thereal-time image that the user wishes to remove or make transparent bydesignating the colors as chroma elements.

The chroma-key mixer 28 may perform the above-described process throughvarious techniques. For example, in one embodiment of the invention, thechroma-key mixer 28 utilizes digital processing to create the compositeimage. In other embodiments, the chroma-key mixer 28 may create thecomposite image through optical techniques or through the use of analogreal-time circuits that are known in the art. In yet other embodiments,the chroma-key mixer 28 comprises a luminance key mixer, which performsvideo compositing based on the brightness of portions of an imageinstead of color.

In the embodiment of the invention depicted in FIG. 2, the chroma-keymixer 28 outputs the composite signal to the switcher 30. The switcher30 also receives the output signal of the multiplexer 26. The functionof the switcher 30 is to select a single output signal from multipleinput signals. For example, in one embodiment, the switcher 30 selectsbetween the signal from the multiplexer 26 and the composite signal fromthe chroma-key mixer 28. The switcher 30 makes its determination basedon communications with the control circuitry 20. In particular, theoperation of the switcher 30 is managed by the control circuitry 20based on the information contained in the data files, such as referenceinformation regarding the beginning and ending reference points.

In one embodiment, the control circuitry 20 cross-references each frameof a prerecorded video with the beginning and ending reference pointscontained in the data files corresponding to the specific video beingplayed. When the DVD counter, whose value relates to the specific framebeing played, matches or correlates with a beginning reference pointvalue, the switcher 30 is automatically instructed to select thecomposite signal from the chroma-key mixer. As a result, a target imagebeing captured by the video recorder 16 is “inserted” or superimposed inthe prerecorded video scene. When the DVD counter matches or correlateswith an ending reference point, the switcher 30 is automaticallyinstructed to select the signal from the multiplexer 26, thus removingthe image of the target subject from the prerecorded video scene.

For example, suppose a user wants to insert himself or herself for DarthVader® in a video clip from Star Wars, which is being played by the DVDplayer 24. First, the data files for the movie Star Wars that containinformation relating to the video footage of Darth Vader® are accessedby the control circuitry 20. During frames not containing video footageof Darth Vader®, the signal from the multiplexer 26 (which comes fromthe DVD player 24) is selected by the switcher 30, which signal may beshown by the display 18. Viewers of the display 18 will see the normalfootage from the Star Wars movie. The signal from the multiplexer 26 isselected by the switcher 30 until the control circuitry 20 instructs theswitcher 30 to select the composite signal from the chroma-key mixer 28.This switching to the composite signal occurs when video footage ofDarth Vader® is contained in the video source signal. The controlcircuitry 20 can identify the footage containing Darth Vader® bycross-referencing the relevant beginning and ending reference pointsfrom the data files. The beginning reference points identify the pointsor times in the movie when Darth Vader® enters a scene. As a result,when a beginning reference point matches or correlates with the DVDcounter, which identifies a particular point of time or frame in themovie, the control circuitry 20 instructs the switcher 30 to select asan output the composite signal from the chroma-key mixer 28. Instead ofseeing Darth Vader® on the display 18, viewers see in his place theimage of the real-time target subject, which is being captured by thevideo recorder 16.

The ending reference points identify the points or times in a movie whenDarth Vader® leaves a movie scene. When an ending reference pointmatches or correlates with the DVD counter, the control circuitry 20then instructs the switcher 30 to select as an output the signal fromthe multiplexer 26. As a result, the target image, which is beingcaptured by the video recorder 16, is not shown on the display 18.

The reference information, therefore, is used in one embodiment of theinvention to automatically control the switching process between thesignal from the video source 14 and the composite signal from thechroma-key mixer 28. For example, in one embodiment of the invention thereference information comprises beginning and ending reference pointsthat correspond to the presence of a particular character in a movie orthat indicate other points when it would be desirable to superimpose areal-time target image on a prerecorded image. The reference informationof the data files may also be used to manage the audio components of thesignals received from the multiplexer 26 and the video recorder 16. Forexample, in one embodiment, the control circuitry 20 instructs theswitcher 30 to: (1) include only the audio component of the signal fromthe multiplexer 26 in the output signal, (2) include only the audiocomponent of the signal from the video recorder 16 in the output signal,or (3) include both the audio components of the signals from themultiplexer 26 and the video recorder 16 in the output signal.

In another embodiment, the reference information is also used to managethe display of voice prompts, such as are used in karaoke files. Forexample, the reference information may indicate when to show voiceprompts for a particular character when the character enters a scene.The reference information may also indicate when to remove or notdisplay voice prompts for the particular character. In such anembodiment of the invention, the reference information corresponding tovoice prompts may be located in the same data file as, or in a separatedata file from, the reference information corresponding to video oraudio components of the video source.

Although, the terms “beginning reference points” and “ending referencepoints” are used herein to describe the functioning of the compositingsystem 10, one with skill in the art will recognize that the beginningand ending reference points may be structurally and functionallyequivalent. For example, in one embodiment, reference points stored inthe data files are not identified as beginning or ending referencepoints. The reference points may be used by the control circuitry 20 tooutput a signal that causes the switcher 30 to change its state nomatter what state the switcher was operating in previously.

With continued reference to FIG. 2, the depicted embodiment of thecompositor device 12 comprises the user interface 32. The user interface32 comprises any interface that accepts input from a user and/or conveysinformation to a user. In one embodiment, the user interface 32 iscoupled to the chroma-key mixer 28 and to the control circuitry 20. Inother embodiments, the user interface 32 may be coupled to more or fewercomponents of the compositor device 12. For example, the user interfacemay be directly coupled to the DVD player 24 to control the operation ofthe DVD player without the use of the control circuitry 20.

One embodiment of the user interface 32 is illustrated in FIG. 3. Theuser interface 32 comprises a front tray portion of the DVD player 24, adisplay 40, editing controls 42, cropping/chroma controls 44 and acamera input display 46. In other embodiments of the invention, the userinterface 32 may comprise more or fewer components. For example, theuser interface 32 may operate without the display 40 or without theediting controls 42.

The display 40 conveys to the user information regarding the operationof the compositor device 12. For example, the display 40 may depictinformation regarding the tracks of an inserted DVD (either by time orby frame number), the chroma color selections, the data files (such asthe film title or the tracks/scenes available for substitution) andother information to assist the user. In one embodiment, the display 40is a light emitting diode (LED) display. In other embodiments, thedisplay 40 is a liquid crystal display (LCD). The display 40 may conveyinformation through the use of graphics, characters, or both.

The cropping/chroma controls 44 allow a user to allow a user to modifyin real time the video image being captured by the video recorder 16 sothat the image conforms to the prerecorded video scene in which theimage is inserted. In one embodiment, the cropping/chroma controls 44allow the user to select the chroma element or elements to be subtractedfrom the captured video image. Such selection may be made by selectingthe name of a particular color or by selecting a visual representationof the color that is shown on the user interface display 40 or theexternal display 18.

The cropping/chroma controls 44 also allow the user to crop the capturedvideo image such that it “fits” in the prerecorded background image.These controls may be used to zoom out or zoom in on a target subject inorder to adjust the size of the target subject to be in proportion withother objects in the prerecorded scene on to which the target subject issuperimposed.

Other embodiments of the invention may contain other controls formodifying the captured video. For example, the user interface 32 maycomprise a color saturation control that adjusts the color level of thecaptured video. This allows for a color image to be adjusted so as toblend in a black and white background.

The camera input display 46 identifies the video recorders 16 that areconnected to the compositor device 12 and that are available to capturevideo for processing. For example, if multiple video recorders 16 werecoupled to the compositor device 12, then multiple lights of the camerainput display 46 may be illuminated. In other embodiments of theinvention, the camera input display 46 identifies when the video beingcaptured by one of the video recorders 16 is being processed and outputto the display 18. In other embodiments of the invention, the userinterface 32 does not include the camera input display 46.

In one embodiment, the user interface 32 also comprises controls thatare generally found on CD/DVD players. For example, the user interfacecomprises a power button 48 to turn the machine on and off. The userinterface also comprises DVD/CD controls 50, such as play, rewind, fastforward, stop, pause, eject and the like, that are used to control theoperation of the DVD player 24.

In one embodiment, the user interface 32 also includes a remote controlinput (not shown). The remote control input may accept instructions ordata that are transmitted to the user interface 32 from one or moreremote control devices. These instructions may correspond to controlsthat are present on the user interface 32 or may include more or fewerinstructions that enable the user to manage the operation of thecompositor device 12.

Though FIG. 3 depicts one implementation of the user interface 32, othertypes of user interfaces may be used in other embodiments. For example,the user interface 32 may comprise a touch screen that both displaysinformation to a user and accepts input from the user. In otherembodiments, the user interface 32 may accept instructions through voicerecognition or may be coupled to another system or device, such as akeyboard or personal computer, that accepts input from a user. In yetother embodiments, the compositor device 12 operates without a userinterface 32. In such embodiments, a user interface may be incorporatedinto the display 18. For example, display 18 may show on-screeninstructions to a user or accept commands inputted by the user.

As shown by the embodiment of the compositor device 12 in FIG. 2, thecontrol circuitry 20 manages the content of the signal outputted to thedisplay 18. In one embodiment, the output of the switcher 30 is coupledto the control circuitry 20. In such an embodiment, the controlcircuitry 20 may select to output the signal from the switcher 30 or mayoutput other video content that is stored in the memory 22. For example,in one embodiment, prerecorded scenes are stored in the memory 22, andupon input from the user, the control circuitry 20 outputs these scenesinstead of the signal from the switcher 30. In another embodiment, thecontrol circuitry 20 is used to overlay information such as subtitles ordialogue prompts over the signal from the switcher 30. In yet otherembodiments of the invention, the output of the switcher 30 is coupledto the display 18 without intervention by the control circuitry 20.

In yet other embodiments of the invention, the compositor device 12includes input/output ports (not shown) that interface with externalsystems or devices. For example, the compositor device 12 may comprise aserial port, a universal serial bus (USB), a firewire port, or the likethat can be used to download data files from a personal computer orhandheld device.

FIG. 4 is a block diagram of an embodiment of the invention wherein thecompositor device 12 is structured to receive input from multiple videorecorders. As can be seen in FIG. 4, the compositor device 12 includesthe same components as the embodiment of the compositor device 12depicted in FIG. 2. The compositor device 12 of FIG. 4 additionallycomprises a second chroma-key mixer 54 and a second switcher 56.

In one embodiment, the second chroma-key mixer 54 functions similarly tothe first chroma-key mixer 28. In particular, the second chroma-keymixer 54 receives, as inputs, signals from the multiplexer 26, the firstchroma-key mixer 28 and a second video recorder 16. The secondchroma-key mixer 54 may also receive instructions from the controlcircuitry 20. Like the first chroma-key mixer 28, the second chroma-keymixer 54 removes the chroma element from a real-time image, such ascaptured by the second video recorder, and combines the modifiedreal-time image with another signal to form a composite signal. In anembodiment having two video recorders, the second chroma-key mixer 54may combine the real-time image captured by the second video recorderwith either the prerecorded video from the multiplexer 26 or thecomposite signal outputted by the first chroma-key mixer 28. The secondchroma-key mixer 54 then outputs a second composite signal to the secondswitcher 56.

In other embodiments, the second chroma-key mixer 54 may be external tothe compositor device 12. In yet other embodiments, portion of thesecond chroma-key mixer 54 may be external to the compositor device 12and portions of the second chroma-key mixer 54 may be internal to thecompositor device 12.

The second switcher 56 functions similarly to the first switcher 30. Inone embodiment, the second switcher 56 receives, as inputs, signals fromthe first switcher 30 and from the second chroma-key mixer 54. Thesecond switcher 56 selects between these inputs based upon instructionsreceived from the control circuitry 20. The second switcher 56 mayoutput a signal to the control circuitry 20 or to the display 18. In oneembodiment of the invention, the content of the output signal of thesecond switcher 56 may be: (1) the prerecorded signal from themultiplexer 26, (2) the composite signal from the first chroma-key mixer28 having portions of an image from the first video recorder, (3) thesecond composite signal from the second chroma-key mixer 54 havingportions of an image from the second video recorder, or (4) the secondcomposite signal from the second chroma-key mixer 54 having portions ofimages from the first video recorder and from the second video recorder.

The operation of one embodiment of the video compositing system will nowbe described. FIG. 5 illustrates one embodiment of an interactive videocompositing process 100. The compositing process 100 begins with State105. At State 105, the user selects a video source that the user wantsto use as a background image for a final composite image. For example,the user may insert a favorite movie into the DVD player 24.

After the user has selected a particular video source, the compositingprocess 100 continues with State 110. At State 110, data filesassociated with the video source are accessed by the compositor device12. For example, when a DVD is inserted into the DVD player 24, theserial number of the DVD is communicated to the control circuitry 20,which uses the serial number to identify and access the appropriateprerecorded data files. The data files identify scenes recorded on theDVD that have been catalogued as being suitable for chroma keysubstitution. In one embodiment of the invention, the suitable scenesare identified by the data files stored in the memory 22 of thecompositor device 12. These data files may have been preprogrammed inthe memory 22, may have been downloaded or saved from another source, ormay be saved on the selected DVD, such as in the bonus materials sectionof the DVD. In one embodiment, the scenes catalogued as being availablefor chroma-key substitution generally contain video footage of acharacter for which a user can substitute his or her own image. In oneembodiment, the data files comprise reference information thatidentifies which scenes of the DVD contain video footage of theparticular character.

The available scenes are communicated to the user, such as through thedisplay 40 of the user interface 32. In another embodiment, theavailable scenes may be communicated through an external display, suchas display 18. At State 115, the user selects an available scene intowhich the user wants to superimpose or “insert” a real-time image. Inone embodiment, the user makes the selection through the user interface32, such as through a remote control.

After the user selects an available scene, the compositing process 100proceeds with State 120. At State 120, the video recorder 16 is used tocapture a target image. This target image is the image that is used tooverlay, or be inserted into, in real time, the scenes from the videosource. The target image is preferably positioned in front of an evenlylit, solid colored background, which color represents the chromaelement. For example, in one embodiment of the invention wherein theselected chroma element is green, the target image is positioned infront of a “green screen.” Other colors or types of backgrounds may beused that enable the background to be later “removed” when forming acomposite image with scenes from the video source.

In one embodiment, a video camera is utilized to capture the targetimage and convert the image into a signal that is input into thechroma-key mixer 28 of the compositor device 12. It should be recognizedthat, along with a video image, an audio signal may also be recorded bythe video camera and input into the compositor device 12.

The chroma-key mixer 28 of the compositor device 12 creates a compositeimage through the processes that have been previously discussed. In oneembodiment, the chroma-key mixer 28 removes the chroma element from theimage captured by the video camera, leaving only the video image of thetarget image. Then, corresponding portions of the image from the videosource are used to fill in the removed portions of the video recordedimage, thus forming a composite image that appears to have the targetimage inserted into scenes from the video source.

In another embodiment, the chroma element portions of the video-capturedimage are made transparent, such as through digital processing, leavingonly the target image. The modified video-captured image is then layeredon top of the video source image. As a result, portions from thebackground video source image that are located underneath the top layertarget image are not seen on the display of the composite image.However, portions of the background video source image locatedunderneath the transparent portions of the top layer (those portionsthat had contained the chroma element) are viewable.

The compositing process 100 then moves to State 125. At State 125, it isdetermined whether the video source image or the composite image is tobe selected as the output image. In one embodiment, this selection ofthe output image is made by the switcher 30. The control of thisselection may be performed automatically (without user interaction) bythe compositor device 12 using information from the prerecorded datafiles, or the user may control the operation of the switcher 30 throughthe user interface 32. For example, the control circuitry 20, based uponthe beginning and ending reference points contained in the data files,may instruct the switcher 30 when to output the video source image andwhen to output the composite image that has the target image overlaid onthe video source image. In another embodiment, the user may alsomanually control when such switching occurs. Such manual control allowsthe user more freedom to create desired scenes or special effects.

If at State 125, the switcher 30 is instructed to select the compositeimage, the compositing process 100 moves to State 130. At State 130, thecomposite image is shown on the display 18. Viewers of the display 18will observe the real-time target image inserted into the prerecordedfootage from the video source. For example, viewers may see the targetimage replacing a character in a movie playing on the DVD player 24.After State 130, the compositing process 100 moves to State 140.

If at State 125, the switcher 30 is instructed to select the videosource image, the compositing process 100 moves to State 135. At State135, the image from the video source is shown on the display 18. AfterState 135, the compositing process 100 then proceeds to State 140.

At State 140, it is determined whether or not preprogrammed data is tobe displayed instead of the video source image or the composite image.Such preprogrammed data may include, for example, prerecorded scenesthat are stored in the memory 22. In one embodiment, the prerecordedscenes comprise video clips that users may want to insert in order makethe displayed scenes appear more interactive or to appear morelife-like. For example, prerecorded video clips having various forms offeedback from judges could be inserted after a target subject has actedout a scene (which was observed by the viewers of the display 18). Inone embodiment, different video clips are selected to be displayed basedon input given by the viewers.

If preprogrammed data is to be displayed, the compositing process 100moves to State 145. At State 145, the preprogrammed data is communicatedto the display 18. In one embodiment, the control circuitry 20 manageswhich signal is communicated to the display 18. In another embodiment, amultiplexer or other similar device may be used to select which signalis output to the display 18. The length of time that the preprogrammeddata is displayed may be directly controlled by the user or may be a setlength of time, such as, for example, until the end of the particularvideo clip. Upon completion of State 145, the compositing process 100returns to State 140. If preprogrammed data is not to be displayed, thecompositing process 100 returns to State 125 to determine whether thevideo source image or the composite image is to be displayed.

One with skill in the art will recognize that the compositing process100 illustrated in FIG. 5 is only one example of the functioning of thecompositing system 10. For example, in other embodiments of theinvention, the compositing process 100 may include more or fewer states,or states may be combined or executed in a different order. For example,additional states may be added that illustrate the separate control ofaudio signals and video signals. In other embodiments, preprogrammeddata may be displayed at the beginning of the compositing process 100 orupon the selection of a particular video source for playback.

The compositing system 10 may be particularly useful with preprogrammedvideo that is easily adapted to allow for user interaction. For example,to help further illustrate an embodiment of the compositing process 100,assume that the user selects a DVD that has recorded scenes from thetelevision talent show American Idol. Data files corresponding to videosegments of the DVD are stored in the memory 22 of the compositor device12. Upon insertion of the DVD, the user is provided with options ofscenes that are available for user interaction. For example, the usermay have the option to select different scenes in which the user may“perform” in front of the judges or an audience.

Through the user interface 32, the user selects a scene for videocompositing. For example, the user may pick a scene in which acontestant is performing by signing a song in front of the judges. Auser whose image is going to be substituted into the American Idolfootage is positioned in front of the video recorder 16 with a greenscreen serving as the background. The individual then performs as if heor she was actually participating on the American Idol program, theperformance of the individual being captured by the video recorder 16and converted to a signal communicated to the compositor device 12.

The display 18 shows video and audio from the American Idol program.During the American Idol scenes that would normally contain footage ofthe participant on the actual program, the image of the individual beingrecorded by the video recorder 16 is substituted in real time for theparticipant. As a result, it appears to viewers of the display 18 thatthe individual is actually participating on the American Idol program.The timing of the substitution of images is determined by the referenceinformation recorded in the data files, which process has been describedpreviously. In addition, the user may also manually control which imagesare output to the display 18.

Audio signals that are captured by the video recorder 16 are also outputthrough the display 18. Like the video images, the substitution of thereal-time audio signals from the video recorded footage may occur atappropriate points in the American Idol scenes, such as when theparticipant is performing or singing. The audio substitution need notoccur at the same times as the video substitution. For example, theremay exist portions of American Idol footage that contain the voice ofthe participant but that do not contain the video image of theparticipant. Again, the substitution of the audio signals may beautomatically controlled by the compositor device 12 based on the datafile information and/or may be manually controlled by the user.

After the performance, viewers have the option to rate the performanceof the individual who has been inserted into the program. These viewerratings may be used to select the display of prerecorded video clipshaving feedback from the judges on American Idol. For example,prerecorded video clips of good reviews, bad reviews, and averagereviews may be stored in the memory 22 of the compositor device 12. Theviewers then have the option of inputting their opinions of theperformance, such as through remote controls communicating with the userinterface 32. If the viewers rate the performance by the individual asbeing generally poor, then the compositor device 12 selects the playbackof video clips that include the judges being critical of theperformance. On the other hand, if the users rate the performance asbeing generally good, then the compositor device 12 selects the playbackof video clips that give positive feedback from the judges.

In other embodiments of the invention, the above-described American Idolprogram may be stored entirely in the memory 22 of the compositor device12 without the use of a DVD. In yet other embodiments, the compositordevice 12 may include a monitor that display the appropriate voiceprompts to the user, which prompts correspond to portions of theAmerican Idol program. For example, the compositor device 12 may includea CD+G input or player that reads files and outputs a display havingvoice prompts.

Although embodiments described herein have related generally toreal-time video compositing with sources such as movies, televisionprograms and the like, other embodiments of the invention may utilizeany source of video, audio, or a combination thereof. For example, inanother embodiment of the invention, the compositor device may acceptvideo or audio input from a video game system. Such would allow a userto “insert” himself or herself into the video game and to interact withobjects therein.

While certain embodiments of the inventions have been described, theseembodiments have been presented by way of example only, and are notintended to limit the scope of the inventions. Indeed, the novel methodsand systems described herein may be embodied in a variety of otherforms; furthermore, various omissions, substitutions and changes in theform of the methods and systems described herein may be made withoutdeparting from the spirit of the inventions.

1. A system for providing real-time, interactive entertainment, the system comprising: a first input that receives a first video signal comprising video content having a plurality of frames, the plurality of frames representing a cinematic scene having first and second original characters and a first audio signal comprising audio content for the cinematic scene; a second input that receives a second video signal comprising a real-time video image of a participant and a second audio signal associated with the participant; a mixer module in communication with the first input and with the second input, wherein the mixer module outputs a combined signal comprising portions of the first video signal and portions of the second video signal, wherein the portions of the second video signal comprise the real-time video image of the participant, and wherein the combined signal is configured to cause the display of the real-time video image of the participant in place of the original character such that the first original character in the cinematic scene is replaced with the participant; a data file associated with the first video signal, the data file comprising, first reference data that identifies a first frame of the plurality of frames in which the first original character enters the cinematic scene of the video content, and second reference data that identifies a second frame of the plurality of frames in which the first original character exits the cinematic scene of the video content, and a switching module that selects between outputting the combined signal and outputting the first video signal based on the data file, wherein the switching module outputs the combined signal at a time identified by the first reference data whereby the participant enters the cinematic scene in place of the first original character, and wherein the switching module outputs the first video signal at a time identified by the second reference data whereby the participant exits the cinematic scene, and wherein the switching module outputs at least the second audio signal at the time identified by the first reference data and outputs at least the first audio signal at the time identified by the second reference data.
 2. The system of claim 1, wherein the switching module performs said selection automatically in real time.
 3. The system of claim 1, wherein the data file further comprises third reference data that identifies a third frame of the plurality of frames in which the first original character re-enters the cinematic scene of the video content, and wherein the switching module outputs the combined signal at the time identified by the third reference data whereby the participant re-enters the cinematic scene in place of the first original character.
 4. A reference data system used for video compositing, the system comprising: first reference information associated with a first video signal and a first audio signal, the first video signal comprising video content having a plurality of frames representing a cinematic scene having at least two characters and the first audio signal comprising audio content for the cinematic scene, wherein the first reference information identifies a first frame of the plurality of frames in which a first of the at least two characters appears in the cinematic scene represented by the video content; second reference information associated with the first video signal and the first audio signal, wherein the second reference information identifies a second frame of the plurality of frames in which the first character no longer appears in the cinematic scene represented by the video content; and a switcher module configured to switch between outputting either (i) the first video signal and the first audio signal or (ii) a combined video signal that comprises portions of the first video signal and portions of a second video signal and at least a second audio signal, the second video signal comprising a real-time video image of a participant and the second audio signal comprising audio content associated with the participant, and wherein the combined video signal is configured to cause the display of the real-time video image of the participant in place of the first character in the cinematic scene, wherein the switcher module outputs the combined video signal and at least the second audio signal at a first time identified by the first reference information whereby the participant appears in the cinematic scene in place of the first character, and wherein the switcher module outputs the first video signal and at least the first audio signal at a second time identified by the second reference information whereby the participant no longer appears in the cinematic scene.
 5. The reference data system of claim 4, wherein the switcher module is configured to receive at least one of the first video signal and the second video signal via the Internet.
 6. The reference data system of claim 4, wherein the switcher module comprises a processor configured to digitally process the first and second video signals to create the combined video signal.
 7. A system for providing interactive entertainment, the system comprising: at least one first input for receiving a first video signal comprising video content including a plurality of frames, the plurality of frames representing a scene having a first character and a second character, and a first audio signal comprising first audio content for the scene; at least one second input for receiving a second video signal comprising a real-time image of a participant and a second audio signal comprising second audio content associated with the participant; and a processor in communication with the at least one first input and the at least one second input, wherein the processor is configured to combine a portion of the first video signal and a portion of the second video signal to form a combined signal, wherein the portion of the second video signal comprises the real-time image of the participant, and wherein the combined signal comprises video information with the participant as the first character in the scene, wherein the processor is further configured to receive first reference data that identifies a first frame of the plurality of frames in which the first character appears in the scene of the video content and to receive second reference data that identifies a second frame of the plurality of frames in which the first character exits the scene of the video content, wherein the processor is further configured to output the combined signal and at least the second audio content at a first time identified by the first reference data such that the participant appears in the scene as the first character and to output the first video signal and at least the first audio content at a second time identified by the second reference data such that the participant no longer appears in the scene.
 8. The system of claim 7, wherein at least one of the first and second inputs is configured to receive, respectively, the first or second video signals via the Internet.
 9. The system of claim 7, wherein the processor is configured to perform chroma-key processing of the second video signal to isolate the real-time image of the participant from a background of the second video signal. 